Why Diversity Is Hard